Skip to content

Conversation

@stmh
Copy link
Member

@stmh stmh commented Sep 11, 2025

No description provided.

stmh added 18 commits August 31, 2025 21:55
Hide custom actions dropdown and divider when user lacks manage
permissions or when no custom actions are available. This prevents
UI clutter and ensures users only see actions they can actually use.

- Add canManage prop to CustomActionsDropdown component
- Export hasActions reactive binding to control divider visibility
- Only show divider when both regular actions and custom actions exist
- Simplify dropdown logic to only render when actions are available
- Remove placeholder for missing actions (cleaner UI when no actions)
- Add permission loading state management to prevent buttons disappearing on page reload
- Export permissionsLoading store for UI loading states
- Add loadUserPermissions call in onMount to ensure permissions are loaded
- Fix reactive statement ordering by inlining availableActions calculation
- Add loading UI while permissions are being fetched
- Add comprehensive debug logging for permission state tracking

This resolves the issue where action buttons would vanish after page reload
due to race conditions in permission loading and reactive statement execution.
- Fix field reassignment warnings in apps list handler
- Add allow attributes for future-use fields (picture, effective_permissions)
- Remove unused authorization service methods (format_identifier_user_id, get_app_scopes)
- Keep debug_authorization_state method with allow attribute for debugging
- Replace brittle path parsing with Axum MatchedPath extractor
- Fix permission bypass when authorization is not configured
- Replace server panic with fallback to default configuration
- Fix OAuth tests to use proper fallback service configuration

Security improvements:
* Authorization middleware now denies requests when not properly configured instead of allowing all
* New AuthorizationNotConfigured error provides clear feedback
* Server gracefully falls back to default config instead of crashing
* Path extraction uses route templates instead of hardcoded parsing

Technical improvements:
* Uses MatchedPath to extract {app_id} and {app_name} parameters robustly
* Maintains backwards compatibility with legacy path parsing
* Added comprehensive tests for template-based extraction
* Fixed OAuth tests to create fallback service with test assignments
- Fix immutable reactive statement in custom-actions-dropdown.svelte
- Remove unused getAvailableActions function in dashboard page
- Remove unused  variable in permissionStore.ts

All frontend code now passes ESLint and Prettier validation.
- Add prominent security warnings about never storing actual tokens in config files
- Add comprehensive 'Bearer Token Security Best Practices' section in configuration.md
- Update examples to use placeholder values and environment variable overrides
- Enhance authorization.md with secure token configuration examples
- Add security note to main README with reference to detailed best practices
- Update migration guide with step-by-step secure migration process

Security improvements:
* Clear guidance on using environment variables for actual tokens
* Strong token generation examples using openssl rand -base64 32
* Token rotation procedures and access control principles
* Example secure deployment scripts with proper file permissions
The test environment was using role-based subject matching g(r.sub, p.sub)
while production used direct string matching r.sub == p.sub, causing
inconsistent authorization behavior.

Changes:
- Update test model matcher from g(r.sub, p.sub) to r.sub == p.sub
- Ensures consistent authorization enforcement across all environments
- All tests verified and passing with corrected model
- Enhanced error logging in basic_auth.rs to include request URL, method, and user agent for better debugging
- Fixed login handler to properly validate bearer tokens against RBAC assignments using reverse lookup
- Updated frontend authentication to properly await token validation and handle errors
- Fixed validate-token calls on login page to include Authorization header
- Updated Tailwind CSS dependencies to compatible v4.1.12 to resolve build issues
- Added comprehensive unit tests for login endpoint scenarios (though config issues prevent full test execution)
- Verified all authentication flows work correctly via manual curl testing

The authentication system now properly validates tokens, provides better error context, and handles all auth modes correctly.
…rvice

The get_user_by_token method was never called and has been superseded by
get_user_by_identifier which handles the new identifier format.
# Conflicts:
#	Cargo.lock
#	frontend/package.json
Introduces comprehensive design document for addressing time-synchronicity
issues in current stdout/stderr handling and enabling real-time log streaming
and interactive shell access to Docker services.

Key features planned:
- Unified time-synchronized output preserving chronological order
- Real-time log streaming via app:logs command with WebSocket support
- Interactive shell access via app:shell command
- Bollard-based Docker integration with docker-compose fallback
- Memory management with configurable output limits
- Permission-based access control for logs and shell operations
- Frontend integration planning for unified log viewer

Breaking changes documented for TaskDetails stdout/stderr removal.
Completes technical validation of bollard capabilities for unified output system.

Technical spike results:
- Validates bollard container logs API with real-time streaming
- Confirms exec API supports interactive shell sessions
- Tests stdout/stderr separation and timestamp handling
- Proves service discovery integration with existing patterns

Key findings:
- Bollard provides all required capabilities for logs and shell access
- Existing service discovery via docker labels works perfectly
- WebSocket integration path is clear and low-risk
- Performance benefits over docker-compose subprocess approach

Files added:
- docs/technical-spike-bollard-findings.md: Comprehensive analysis and recommendations
- scotty/examples/bollard_spike.rs: Working proof-of-concept with real container testing

Ready for implementation Phase 1: Core Infrastructure.
@claude
Copy link

claude bot commented Sep 17, 2025

Claude finished @stmh's task —— View job


Code Review for Version 0.2.0 PR

Review Checklist:

  • Read repository guidelines and understand project structure
  • Review authentication and authorization system changes
  • Examine core library (scotty-core) modifications
  • Review server-side (scotty) changes
  • Check frontend updates and new components
  • Assess configuration and deployment changes
  • Evaluate test coverage and quality
  • Check for security concerns
  • Review performance implications
  • Provide final recommendations

Summary

This is a substantial and well-architected release that introduces a comprehensive authentication and authorization system to Scotty. The implementation demonstrates good security practices and clean code organization.

Positive Aspects

🏗️ Architecture & Design

  • Excellent separation of concerns: Authentication (who you are) vs authorization (what you can do)
  • Flexible auth modes: Development, OAuth, and Bearer token modes with clean configuration
  • Casbin RBAC integration: Industry-standard authorization with roles, scopes, and permissions
  • OAuth compliance: Proper RFC 6749 error handling and device flow support

🔐 Security Implementation

  • Proper token handling: Bearer tokens are validated and mapped to identifiers securely
  • OAuth error handling: Comprehensive error types with appropriate HTTP status codes
  • Authorization middleware: Proper permission checking with fallback mechanisms
  • OIDC validation: Native OAuth token validation without relying on proxy headers

🧪 Test Coverage

  • Integration tests: Comprehensive OAuth flow and bearer auth testing
  • Mock servers: Proper OAuth provider mocking for isolated testing
  • Auth scenarios: Multiple authentication modes tested

Issues & Recommendations

🔴 Security Concerns

1. Token Storage in localStorage (High Priority)

// frontend/src/routes/oauth/callback/+page.svelte:66
localStorage.setItem('oauth_token', tokenData.access_token);

Issue: OAuth access tokens stored in localStorage are vulnerable to XSS attacks and persist across browser sessions.

Recommendation:

  • Use httpOnly cookies for token storage when possible
  • Implement token expiration handling
  • Consider using sessionStorage for shorter-lived sessions
  • Add CSP headers to mitigate XSS risks

2. Token Logging Risk (Medium Priority)

// scotty/src/api/basic_auth.rs:163
warn!("OAuth authentication failed - invalid Authorization header format (expected 'Bearer <token>', got: {}...)", 
      auth_header.chars().take(20).collect::<String>());

Issue: Partial token values in logs could aid in token enumeration attacks.

Recommendation: Log only token prefix (first 4-6 characters) or use token hash for debugging.

🟡 Code Quality Issues

3. Panic-Prone Code (Medium Priority)

// scotty/src/settings/config.rs:171
let settings: Settings = builder.build().unwrap().try_deserialize().unwrap();

Issue: Multiple unwrap() calls in configuration loading could cause panics.

Recommendation: Use proper error handling with ? operator and contextual errors.

4. TODO Items (Low Priority)
Found several TODO items including:

  • scottyctl/src/api.rs:29: Token refresh logic
  • scotty/src/oauth/handlers.rs:556: JWT token generation
  • Authorization assignment removal

Recommendation: Track these in issues and prioritize for future releases.

🟠 Performance Considerations

5. Permission Checking Overhead

// Authorization middleware runs on every request
let effective_permissions = auth_service.get_user_permissions(&user_id).await;

Recommendation:

  • Implement permission caching to reduce database/config lookups
  • Consider JWT-based permissions for stateless authorization

6. RBAC Configuration Loading
The Casbin policy loading from YAML could benefit from caching mechanisms for production deployments.

Minor Improvements

📦 Dependencies

  • Good use of established crates (Casbin, Axum, Tokio)
  • Appropriate feature flags in Cargo.toml
  • Version management utilities are well-designed

🎨 Frontend

  • Clean Svelte store architecture for user state
  • Proper reactive patterns for authentication state
  • Good separation of auth modes in UI

🔧 Configuration

  • Comprehensive OAuth configuration options
  • Good use of environment-based settings
  • Clear example configurations

Final Verdict

✅ Approve with Recommendations

This is a high-quality implementation that significantly enhances Scotty's security and user management capabilities. The architecture is sound, the code is well-structured, and the test coverage is comprehensive.

Priority Actions:

  1. Address token storage security (localStorage → httpOnly cookies or sessionStorage)
  2. Reduce token logging exposure (log only prefixes/hashes)
  3. Replace unwrap() calls with proper error handling
  4. Add permission caching for better performance

The benefits of this release far outweigh the issues identified, which are typical for a major authentication overhaul and can be addressed in follow-up PRs.

Estimated Effort for Fixes: 1-2 days for high-priority security issues, 2-3 days for performance optimizations.

stmh added 11 commits September 17, 2025 15:47
- Add OutputLine, TaskOutput, and OutputStreamType structures for time-synchronized output collection
- Add OutputSettings for configurable limits on lines and line length
- Add ShellSettings for shell session configuration (TTL, limits, defaults)
- Integrate settings into main configuration structure
- Support both default limits and custom settings per task
BREAKING CHANGE: Remove stdout and stderr fields from TaskDetails

- Remove stdout/stderr string fields from TaskDetails structure
- Remove print/println methods from TaskDetails
- Add TaskOutput to TaskState for unified output collection
- Update TaskManager to use unified output streams with configurable limits
- Add methods to TaskManager for adding messages to task output
- Replace separate stdout/stderr readers with unified stream handlers
- Update CLI to show basic task status (TODO: implement full output streaming)
- Update wait_for_all_containers_handler to add status messages to task output
- Replace server-only logging with client-visible task messages
- Messages now appear in UI instead of only server logs
- Maintain server logging for debugging purposes
- Add note about running cargo fmt and cargo clippy for code quality checks
- Add LogsStreamInfo, LogsStreamData, LogsStreamEnd, LogsStreamError for log streaming
- Add ShellSessionInfo, ShellSessionData, ShellSessionEnd, ShellSessionError for shell sessions
- Add ShellDataType enum for input/output differentiation
- Extend WebSocketMessage enum with comprehensive real-time streaming support
- Messages support stream/session identification via UUIDs
- Add unified output system implementation progress to CLAUDE.md
- Document completed Phase 1 tasks and next Phase 2 objectives
- List key modified files and reference documents for easy resumption
- Add LogStreamingService for real-time container log streaming via WebSocket
- Add ShellService for interactive container shell sessions with TTY support
- Support session limits, TTL, and input/output streaming
- Handle Docker timestamp extraction for proper log formatting
- Implement resize support for TTY sessions
- Both services use bollard Docker API for container interaction
Add find_container_by_service() and get_container_id_for_service()
helper methods to AppData to simplify container discovery operations
and reduce code duplication across services.
- Replace String errors with proper enum-based error types using thiserror
- Add LogStreamError and ShellServiceError enums with detailed error variants
- Implement helper methods on session structs for better encapsulation
- Add LogOutputConverter and LogBuffer for improved log processing
- Add constructor methods to WebSocket message types
- Improve error messages with structured context
- Add LogStreamError and ShellServiceError variants to AppError enum
- Implement From trait for automatic error conversion
- Map service errors to appropriate HTTP status codes
- Ensure proper error propagation through the API layer
- Add POST /apps/{app_id}/services/{service_name}/logs for log streaming
- Add DELETE /logs/streams/{stream_id} to stop log streams
- Add POST /apps/{app_id}/services/{service_name}/shell for shell sessions
- Add POST /shell/sessions/{session_id}/input for sending input
- Add POST /shell/sessions/{session_id}/resize for TTY resizing
- Add DELETE /shell/sessions/{session_id} to terminate sessions
- All endpoints include proper OpenAPI documentation
- Appropriate permission levels (View for logs, Manage for shell)
stmh and others added 30 commits November 4, 2025 16:34
Fix regression where scottyctl only displayed HTTP status codes instead
of the actual error messages returned by the server in JSON responses.

Changes:
- Add extract_error_message() helper to HttpClient
- Parse JSON error responses and extract "message" field
- Update get_json() to use error message extraction
- Update post_json() to use error message extraction
- Fallback to status code if message extraction fails

Before:
  Error: HTTP error: 400

After:
  Error: 400 Bad Request: App name already exists

This improves user experience by showing meaningful error messages
returned from the scotty server API.
Remove unused helper functions that were causing clippy warnings:
- create_test_app_data() from shell_test.rs
- create_test_app_data() from logs_test.rs
- create_test_websocket_clients() from login_test.rs

Also remove unused imports that were only used by these functions.
Implements Phase 1 of file transfer optimization by adding gzip
compression before base64 encoding. This reduces payload size by
approximately 58% for typical text-heavy projects.

Changes:
- Add 'compressed' boolean flag to File struct for tracking compression state
- Implement gzip compression in scottyctl before base64 encoding
- Add decompression logic in server handler after base64 decoding
- Display compression statistics in CLI output
- Add comprehensive unit tests for compression/decompression

Results from testing with nginx-with-scottyignore example:
- Original size: 9.17 KB
- Compressed size: 3.80 KB (58% reduction)
- Final payload: 17.73 KB (with base64 encoding)

The implementation is backward compatible - the compressed field
defaults to false, so old clients continue to work with new servers
and vice versa.

Related to scotty-e7d3
Add comprehensive test coverage and security enhancements based on PR review:

1. Add 7 unit tests for extract_error_message() covering all code paths
   (JSON with message, plain text, long body, excessive body, empty body,
   JSON without message field, malformed JSON)

2. Add 10KB body size limit to error extraction to prevent DoS attacks
   via large error response bodies. Implements streaming with size checks.

3. Improve .git ignore path handling using path.components() instead of
   string contains check to correctly handle .git at root and nested levels

4. Verify parent directory loop is necessary with new test case and add
   clarifying comment (ignore crate does not auto-handle parent dirs)

5. Add 4 new tests for hardcoded ignore patterns (git at root, git nested,
   .DS_Store, and parent directory ignored)

Dependencies added: wiremock for testing, futures-util for streaming,
enabled stream feature on reqwest

All 154 tests passing. Addresses all critical and recommended issues
from PR review.
feat: add .scottyignore support and fix error message display
Task output was appearing all at once after task completion instead of
streaming in real-time. The issue had two root causes:

Server-side:
- StateMachine::spawn() was awaiting the spawned task, blocking the API
  response until task completion (scotty-44 regression)
- Fixed by removing the .await that was waiting for task execution

Client-side:
- WebSocket connected AFTER the API call, missing early output
- Fixed by connecting WebSocket BEFORE calling API endpoints

Additional improvements:
- Updated task output formatting with consistent 4-char prefixes
- Added stdout/stderr flushing for immediate display
- Simplified error handling (panics only, errors via TaskCompletionHandler)

Changes:
- scotty/src/state_machine.rs: Remove blocking .await from spawn()
- scotty/src/docker/helper.rs: Simplified to return immediately
- scottyctl/src/commands/apps/*.rs: Connect WebSocket before API calls
- scottyctl/src/api.rs: Updated wait_for_task signature
- scottyctl/src/utils/status_line.rs: Added stdout/stderr flushing

The API now returns instantly while tasks run in background with full
real-time output streaming to connected WebSocket clients.
Task output was appearing all at once after task completion instead of
streaming in real-time. The issue had two root causes:

Server-side:
- StateMachine::spawn() was async and appeared to block the API response
- Root cause: spawn_instrumented() was async (acquiring tokio RwLock)
- Fixed by making spawn_instrumented() synchronous using std::sync::RwLock
- This makes StateMachine::spawn() return JoinHandle immediately
- Task spawns in background without blocking the API response

Client-side:
- WebSocket connected AFTER the API call, missing early output
- Fixed by connecting WebSocket BEFORE calling API endpoints

Additional changes:
- Updated all spawn_instrumented() call sites to remove .await
- Updated tests to work with new synchronous spawn() behavior
- Change file processing to fail fast instead of silently dropping files on decompression errors
- Replace unwrap() calls with proper error propagation using ? operator
- Add decompression bomb protection using max upload size from settings (default 10MB)
- Remove unused mut warning in GzDecoder
Address PR feedback with must-fix and should-fix items:

- Add specific error types (FileCompressionCorrupted, FileDecompressedSizeExceeded) for better debugging
- Fix size limit check to properly detect oversized files without accepting truncated data
- Use read(max_size + 1) approach to detect if decompressed content exceeds limit
- Add comprehensive tests for size limit enforcement (both exceeding and within limit cases)

This prevents accepting corrupted/truncated files and provides clearer error messages
for debugging production issues.
feat: add gzip compression for file uploads in app:create
- Add TODO comment in helper.rs explaining state machine handle drop
- Reference scotty-f1dd for future multi-handle support
- Add warning log when WebSocket connection fails in scottyctl
- Helps users debug connectivity issues when real-time output unavailable

Addresses PR feedback about task error handling transparency.
fix: enable real-time task output streaming in scottyctl
…tation

Remove intermediate working documents that are no longer needed:
- Secret handling analysis and migration docs (work completed in PR #498)
- Bollard technical spike findings (implementation complete)

Fix OAuth authentication documentation:
- Correct token storage from localStorage to sessionStorage (3 occurrences)
- Remove oauth2-proxy migration section (never published, development-only)
- Enhance Security Features section with specific implementation details:
  - PKCE: SHA256 challenge, MaskedSecret storage, single-use verification
  - CSRF: session-based tracking, 5-minute cleanup cycle, time-limited sessions
  - Token Security: per-request validation, OIDC provider validation, session expiration times
Add missing permissions and admin API documentation:
- Document admin_read and admin_write global permissions (8 total, was 6)
- Add complete admin API endpoint documentation (scopes, roles, assignments, permissions)
- Add system_admin role example for authorization-only management
- Document separation of authorization management vs app management
- Add system administration example showing least privilege for admin roles
- Update API endpoints section with WebSocket shell sessions (not "future")
- Clarify global vs app-specific permissions

All changes verified against implementation in:
- scotty/src/services/authorization/types.rs (Permission enum)
- scotty/src/api/router.rs (admin routes)
- scotty/src/api/rest/handlers/admin/* (endpoint implementations)
Add missing command documentation:
- Authentication section: OAuth (auth:login/logout/status/refresh) and bearer tokens
- Admin commands: Complete coverage of authorization management
  - admin:scopes:list/create - Manage authorization scopes
  - admin:roles:list/create - Manage roles and permissions
  - admin:assignments:list/create/remove - Manage user assignments
  - admin:permissions:list/test/user - Query and test permissions
- Missing app commands: app:action (custom actions)
- Missing blueprint commands: blueprint:info (blueprint details)
- Shell completion: Generate completion scripts for bash/zsh/fish/PowerShell

Update existing documentation:
- Update introduction to mention authorization management
- Add note about OAuth vs bearer token authentication throughout
- Clarify that examples use explicit flags but OAuth/env vars are preferred

All commands verified against scottyctl implementation (scottyctl/src/cli.rs).
Replace generic create-svelte template with comprehensive documentation
covering SvelteKit 2, TypeScript integration with Rust types, WebSocket
support, development workflow, and deployment options.
…ose files

- Add support for compose.yml and compose.yaml (Docker Compose v2 standard)
- Create shared compose utilities module in scotty-core
- Consolidate compose file finding logic into reusable functions
- Standardize error messages across the codebase
- Align override config file names with their counterparts
Add rustls-tls-native-roots feature to tokio-tungstenite to support
WebSocket Secure connections through Traefik SSL termination.
Implement domain-based role assignments to allow assigning roles to
all users from a specific domain (e.g., @factorial.io) without
manually adding each email address.

Changes:
- Add extract_domain() helper to extract domain from email addresses
- Add validate_domain_assignment() to validate domain patterns
- Add resolve_user_assignments() with precedence logic:
  1. Exact email match (highest priority)
  2. Domain match (fallback if no exact match)
  3. Wildcard (always additive)
- Update authorization methods to use new resolution logic:
  - check_global_permission()
  - check_permission_in_scopes()
  - get_user_scopes_with_permissions()
  - get_user_permissions()

Domain assignment syntax uses @ prefix (e.g., @factorial.io).
Validation ensures domain has at least one dot and no extra @ symbols.

Part of scotty-ead6
The function will be used in task 4 (assignment creation API validation)
…ain assignments

Add comprehensive testing and documentation for domain-based role assignments:

- Add API validation for domain assignment patterns
- Add 4 unit tests for domain extraction and resolution logic
- Add 8 integration tests for end-to-end domain assignment flows
- Update config/casbin/policy.yaml with domain assignment example
- Update AGENTS.md with domain assignment documentation

All 163 tests passing.

Part of scotty-ead6
…multiple @ symbols

Addresses code review feedback:
- Change extract_domain to use find('@') instead of rfind('@')
- Reject emails with multiple @ symbols (e.g., [email protected]@factorial.io)
- Add security test coverage for this edge case

This prevents attackers from bypassing domain-based authorization by
using emails like [email protected]@trusted.io to gain access to
@trusted.io domain assignments.

Fixes PR #556 security concern
… 5321

- Normalize complete email addresses to lowercase in resolve_user_assignments
- Preserve case sensitivity for non-email identifiers (identifier:*)
- Add tests for case-insensitive exact and domain matching
- Add tests for identifier and wildcard pattern validation
- Fix API tests to use existing 'admin' role instead of 'viewer'

Addresses PR feedback about case sensitivity in email matching.
Email addresses are case-insensitive per RFC 5321, so [email protected]
should match [email protected].
…ization

feat: Add domain-based role assignment support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants